Video Summarization with Long Short-Term Memory
نویسندگان
چکیده
We propose a novel supervised learning technique for summarizing videos by automatically selecting keyframes or key subshots. Casting the problem as a structured prediction problem on sequential data, our main idea is to use Long Short-Term Memory (LSTM), a special type of recurrent neural networks to model the variable-range dependencies entailed in the task of video summarization. Our learning models attain the state-of-the-art results on two benchmark video datasets. Detailed analysis justifies the design of the models. In particular, we show that it is crucial to take into consideration the sequential structures in videos and model them. Besides advances in modeling techniques, we introduce techniques to address the need of a large number of annotated data for training complex learning models. There, our main idea is to exploit the existence of auxiliary annotated video datasets, albeit heterogeneous in visual styles and contents. Specifically, we show domain adaptation techniques can improve summarization by reducing the discrepancies in statistical properties across those datasets.
منابع مشابه
Video Summarization with Attention-Based Encoder-Decoder Networks
This paper addresses the problem of supervised video summarization by formulating it as a sequence-to-sequence learning problem, where the input is a sequence of original video frames, the output is a keyshot sequence. Our key idea is to learn a deep summarization network with attention mechanism to mimic the way of selecting the keyshots of human. To this end, we propose a novel video summariz...
متن کاملSupplementary Material: Video Summarization with Long Short-term Memory
– Section 1: converting between different formats of ground-truth annotations (Section 3.1 in the main text) – Section 2: details of the datasets (Section 4.1 in the main text) – Section 3: details of our LSTM-based models, including the learning objective for dppLSTM and the generating process of shot-based summaries for both vsLSTM and dppLSTM (Section 3.4 and 3.5 in the main text) – Section ...
متن کاملShort-term and Long-term Impact of Video-driven Metapragmatic Awareness Raising on Speech Act Production: A Case of Iranian Interme-diate EFL Learners
متن کامل
The effect of intrahippocampal microinjection of Naloxone on short –term and long-term memory in adult male rats
Introduction:The hippocampus is one for the major centers of learning and memory. Role of the opioid system has been investigated and on the other hand receptors related to this system such as mu-opioid receptors (MOR) are extended in the hippocampus. In this study the effect of Naloxone administration as a mu opioid receptor antagonist on passive avoidance memory in adult male rats was i...
متن کاملEffect of Short and Long-Term Treatment with Omega-3 Fatty Acids on Scopolamine-Induced Amnesia
Two omega-3 fatty acids including docosahexaenoic acid (DHA) and eicosapentaenoic acid (EPA) are essential for the physiologic function of neuronal cell membrane. Normal function of neuronal cell membrane requires appropriate composition of fatty in its structure. Present study was designed to compare the effect of short-term and long-term pretreatment with omega-3 fatty acids on scopolamine...
متن کامل